The human brain is made up of a group of neurons connected to one another in a network. When we learn something, new connections are made, or existing connections are modified, between the neurons. Neural networks are massively parallel computing models of the human brain, consisting of many simple processors connected by adaptive weights. Neural networks are being used in artificial intelligence-related applications. The parallel neural networks are often simulated on serial computers, with the processors simulated by program code, and the connections modeled by data.
Neural networks can be either supervised or unsupervised. Supervised neural networks differ from conventional programs in that a programmer does not write algorithmic code to tell them how to process data. Instead, a user `trains` a neural network by presenting training data of the desired input/output relationships. As an extremely simple example, let us assume we need a neural network to determine if an input integer is odd or even. We could present training data as follows to the neural network: "1,ODD", "2,EVEN", "5,ODD", "121,ODD", etc. The neural network would learn that all positive integers equal to or less than 121 that are evenly divisible by 2 are "EVEN", and all other positive integers equal to or less than 121 are "ODD". Therefore, after the training session is over, if a user presented a "7" to the neural network as input data, the output result would be "ODD". Note that if we presented a "-14" as input data to the neural network, it may or may not know that this is an even number, since the training data did not "teach" the neural network anything about negative integers. It may correctly guess that is even, but it will probably let the user know it has some doubts that it has arrived at the correct answer. In addition, the neural network may not know that a number such as 123232123121 is odd, since the training data did not include large numbers. Therefore, the quality of the neural network depends on the quality of the training data.
Unsupervised neural networks can extract statistically significant features from input data. This differs from supervised neural networks in that only input data is presented to the network during training.
There are many different types of computing models which fall under the label `neural networks`. These different models have unique network topologies and learning mechanisms. Examples of known neural network models are the Back Propagation Model, the Adaptive Resonance Theory Model, the Self-Organizing Feature Maps Model, the Self-Organizing TSP Networks Model, and the Bidirectional Associative Memories Model. The neural network field is very dynamic, with new network models being proposed often.
Although neural network models are interesting and theoretically powerful, they are inflexible and difficult to use. Highly skilled programmers with knowledge in neural networks are required to write a specialized application program to incorporate the functions of one neural network model. These custom designed application programs are extremely expensive and limited in function, since they are can only perform one specific computation, like minimizing the distance a salesman travels. If a different computation is required, a different application program incorporating the functions of perhaps another, completely dissimilar neural network model must be written by perhaps a different highly skilled programmer familiar with this neural network model.
If it is desired to run these application programs on a different computer system, the whole application program, including the complex neural network component, must often be rewritten. This is clearly undesirable and a waste of precious programming resources. In addition to being rewritten, the neural network component will have to be retrained as well.
A normal application program written in a variety of computer languages by a normal programmer or user with little or no knowledge in neural networks cannot use the power of one or more neural network models to perform a wide range of computations.
The prior art has failed to adequately address the above problems. The prior art has largely been confined to theoretical and experimental applications that are unsuitable for commercial environments.